This invention relates to machine vision, and particularly to applications of machine vision that exploit color imaging.
Color imaging, and other forms of multi-spectral imaging (e.g., imaging that includes near infra-red electromagnetic energy) is useful for facilitating various machine vision tasks, such as alignment and inspection. For example, see U.S. Pat. No. 6,141,033; Michael, et al., issued Oct. 31, 2000, entitled “Bandwidth Reduction of Multi-channel Images for Machine Vision”.
Color or multi-spectral images are obtained using a color (or multi-spectral) camera. Some color cameras use a prism with multiple dichroic coatings to split an image into multiple single-color images. Each single-color image is acquired simultaneously using a separate respective CCD or CMOS image sensor array. Although this method provides a high-resolution multi-color image at high speed, it is the most costly to implement.
Another method for acquiring a color image is to sequentially filter the light from a scene using a rotating multi-color filter wheel and a single CCD or CMOS array to sequentially acquire each of the different single color images. Although this method is less expensive, and provides a high-resolution color image, it is slow, and also suffers from the disadvantage that each single color (e.g., red green, or blue) image is acquired at a different time.
A third method for acquiring a color image uses a single-chip CCD or CMOS sensor array. All three colors are acquired on the same sensor array at the same time. Although CMOS and CCD sensors detect the entire visible light spectrum (and sometimes the near-infrared as well), incident light must be separated into different spectral bands for color imaging. Separation is accomplished by depositing colored filter materials (usually dyed or pigmented polymers) sequentially on the top surface of the sensor array, arranged in a mosaic pattern known as a color filter array (CFA). Photolithographic patterning of the different color layers produces the desired mosaic array.
Several CFA patterns can be used. A common arrangement is a four-pixel repeat cell 100, called the “Bayer pattern”. Each repeat cell 100 consists of two green (G), one red (R), and one blue (B) filter, as shown in FIG. 1A. This results in an alternating pattern of rows 110, wherein a first row consists of red and green alternating filters, and a second row consists of blue and green alternating filters, as shown in FIG. 1B. Thus, in the Bayer pattern, the pixels are not evenly divided. In fact, there are as many green pixels as there are blue and red combined.
Some single-sensor cameras can use alternatives to the Bayer filter pattern, such as repeating a repeating pattern of red, green, and blue stripes. Some single-chip image sensors don't add up the different values of red, green and blue, but instead subtract values using the typesetting colors cyan, yellow, and magenta.
One advantage of this method is that only one image sensor array is required to acquire a complete color image, and all the color information (e.g., red, green and blue) is recorded at the same moment.
The raw output from a color image sensor with a Bayer filter pattern is a mosaic of red, green and blue (RGB) pixels of potentially different intensity. A single-chip color image sensor using an array with n×n pixel resolution is used to generate a three-color RGB image with n×n pixels for each color plane (or color band). However, on the chip only 25% of the pixels are devoted to blue, and 25% of the pixels are devoted to red, whereas 50% of the pixels are devoted to green. (The greater number of green pixels is intended to simulate the relatively greater sensitivity of the human eye to green light.) Therefore, the resolution of the raw data for each color plane is much less than n×n pixels.
These lower resolution color plane images can be combined to create an apparently full-resolution color composite image through application of a Bayer pattern de-mosaic algorithm. This algorithm derives a red, green, and blue intensity value for each pixel location based on the values of the surrounding pixels. However, no new information is actually added by this step, since this is done by color “interpolation” from nearest-neighbor Bayer cells. For example, the red and blue signals at a green-sensitive pixel are estimated from the red or blue signal values on neighboring red- or blue-sensitive pixels. This interpolation step is equivalent to up-sampling and low-pass filtering.
Although less expensive than using a prism, and faster than sequentially acquiring multiple single-color images, this third approach to acquiring a color image yields an effectively low-resolution color image, since all colors are represented in a single color image. Consequently, using known methods, only low-resolution machine vision tasks can be performed using the color image obtained from a single-chip image sensor.
High-resolution machine vision tasks require high-resolution image data. For example, to get highly accurate results when locating an image of an object using a search vision algorithm, high-resolution input image data is required. Due to the low resolution of each color plane of a single-chip color image sensor, a low-cost standard color camera that incorporates such an image sensor is typically not used for high-resolution machine vision applications.